Deep Generative Adversarial Networks for Compressed Sensing (GANCS) Automates MRI

Magnetic resonance imaging (MRI) suffers from aliasing artifacts when it is highly undersampled for real-time imaging. Conventional compressed sensing (CS) MRI analytics are not however cognizant of the image diagnostic quality, and substantially trade-off accuracy for speed in real-time imaging. To cope with these challenges we put forth a novel CS framework that permeates benefits from generative adversarial networks (GAN) to modeling a manifold of MR images from historical patients. Extensive evaluations on a large MRI dataset of pediatric patients by expert radiologists corroborate that GANCS retrieves images with improved quality and finer details relative to conventional CS and pixel-wise schemes. It also offers around 100 times faster inference than existing CS-MRI schemes. 1 Context and Motivation Owing to its superb soft tissue contrast, (real-time) MRI visualization is of paramount importance for diagnostic and therapeutic guidance in next generation imaging platforms. MRI scans are however quite slow, taking several minutes to acquire clinically acceptable images, possibly deteriorated by motion especially for pediatric patients. As a result, the acquisition typically undergoes significant undersampling leading to a seriously ill-posed linear inverse problems for image reconstruction [1, 2]. The scanner collects the k-space data y = Φx + v with Φ ∈ CM×N capturing the Fourier transform and the coil maps where M N , v accounts for modeling noise, and x is the image of interest. In order to retrieve x from y, conventional CS uses sparsity regularization in a proper transform domain such as Wavelet (WV) [1]. It however demands running several iterations of non-smooth optimization algorithms for convergence, and need manual hyper-parameter tunning to optimize the performance, which hinders real-time MRI. With the abundance of historical scans X := {xk}k=1, and the corresponding (possibly) noisy observations Y := {yk}k=1 we are motivated to pursue a deep learning based approach. Suppose the unknown and complex-valued image x lies in a lowdimensional manifoldM. No information is known about the manifold besides the training samples X and Y . Given a new observation y, the goal is then to recover x. Previous work. Recent attempts exist that automate medical image reconstruction by leveraging historical data; see e.g., [3–8]. By training a neural network to map out aliased images to the goldstandard ones, they gain speed up, but they suffer from blurring artifacts and possible hallucinations. This is mainly due to i) adopting pixel-wise costs that are oblivious to high-frequency texture details, which is crucial for drawing diagnostic decisions; and ii) lack of data consistency. GANs [9–11] are very powerful in modeling manifold of natural images that are perceptually appealing; see e.g., [12– 16]. For natural image super-resolution, GANs retrieve realistic images under 4× upscaling [14, 15]. 2 Method The inverse imaging solution amounts to finding a solution at the intersection of data-consistent images and the plausible high-quality image manifold. To effectively learn the manifold from training ∗The authors are with the Stanford University, Departments of Electrical Engineering, Radiology, and Radiation Oncology. 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA. Figure 1: Representative coronal (1st row) and axial (3rd row) images for a test patient retrieved by fully-sampled (1st), GANCS (η = 0.75, λ = 0.25) (2nd), GANCS (η = 1.0, λ = 0.0) (3th), CS-WV (4th), zero-filling (input) (5th) for 5x undersampling. Table 1: Average SNR (dB), SSIM, inference time (sec), and ROS comparison of various schemes. scheme ZF CS-WV GANCS (η = 1, λ = 0) GANCS (η = 0.75, λ = 0.25) SNR 13.66 18.22 21.4 18.03 SSIM 0.64 0.8 0.84 0.76 inference time 10−3 12.28 0.03 0.03 overall quality 2 4.1 4.7 5 sharpness 2 3.5 4.1 5 samples, one needs to ensure c1) trained manifold contains plausible images; and c2) points on the manifold are data consistent, namely y ≈ Φx, ∀x ∈ M? To address c1) we adopt GANs that are very powerful in estimating a prior distribution of sharp images that are visually plausible [9]. A tandem network of generator (G) and discriminator (D) nets is considered, with an initial estimate x̃ as the input for G (this estimate can be simply obtained as the least-norm solution Φ†y). The G net then projects x̃ ontoM to remove the aliasing artifacts, and subsequently G’s output, say x̌, passes through D net to score 1 if x̌ ∈ X , and 0 otherwise. As per c2, G’s output, i.e., x̌, may be inconsistent with the data y. Thus, x̌ is projected onto the feasible set y = Φx through the affine projection x̂ = Φ†y + (I−Φ†Φ)x̌. This resembles a single iteration of projection onto intersection of convex sets (POCS) [17]. Allowing more iterations amounts to training for multiple copies of the G net, and it turns out to considerably improve the performance as will be reported in an extended version of this work. One issue with GANs however is that it introduces high frequency noise that can hallucinate the images. To discard this noise we adopt a mixture of least-squares (LS) GAN [18] and pixel-wise `1 cost, weighted by λ and η, respectively, to train the generator [19]. A residual net with 5 residual blocks and skip connections is adopted for G, and D is a 8-layer CNN for LS classification. 3 Results T1-weighted contrast-enhanced abdominal image volumes are acquired for 350 pediatric patients. With 330 training (50K slices) and 20 test patients (2K slices), the network is trained to map 5-fold undersampled (zero-filled) images (with artifacts) to fully sampled ones. For a random test patient, representative slices from axial and coronal orientations, retrieved by various methods, are shown in Fig. 1. Note, CS-WV is optimized for the best performance using the FISTA algorithm with BART toolbox [20]. It is apparent that GANCS (η = 0.75, λ = 0.25) better reveals the details of liver vessels with more realistic texture and contrast, while other schemes blurred out the images. More quantitative results such as the average SNR (dB), SSIM, inference time (sec), and radiologists opinion score (ROS), ranged from 0 (poor) to 5 (excellent), about quality and sharpness is also listed in Table I. Notice particularly 100 times speed up, and ROS improvement relative to CS-WV. Details